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*  The  Way  Ahead 
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Problem  Statement 


*  Combatant  Commander’s  Future  Force: 

-Technologically  advanced,  Net-centric,  Dynamicaiiy 
distributed.  Rapidly  generated. 

*  Control  Through  Sustainment  Battie  Command  (SBC): 

-  Provides  a  conduit  for: 

-  Technologically  enhanced  information  centralization  / 
distribution. 

-  Building/sustaining  combat  power 

-  Identifying,  mobilizing,  and  delivering: 

-  the  right  requirements, 

-  to  the  right  locations, 

-  in  the  right  quantities, 

-  at  the  right  time. 

-Absolutely  critical  to  build  and  sustain  the  necessary 
combat  power  during  all  phases  of  combat 
operations. 


What  is  Sustainment  Battle  Command? 


“The  application  of  leadership  and  decision  making  to  the 
planning  and  execution  of  sustainment  operations  in 
support  of  combat  ” 


•  Demonstrates  the  proficiency  of  Logistics  Commanders 
to  support  the  combat  mission. 

•  Characterized  by  ieadership  and  decision  making. 

•  Emphasizes  choices  have  substantiai  impact  upon  the 
avaiiabie  capabiiities  of  the  combatant  commanders. 


The  Future  Force 


*  Future  Combat  System  Brigade  Combat  Team  (FCS 
BCT): 

-  4  Combined  Arms  Battalions  (CAB). 

-  8  FCS  Manned  Combat  Systems. 

-  2  FCS  Unmanned  Combat  Systems. 

-  5  Contemporary  Combat  Systems. 

-  1094  Total  Vehicles. 

•  FCS  Operational  Mode  Summary/Mission  Profile 
(OMS/MP)  (2006): 

-  Maintenance/Reliability  Data  collected  from  Vector  in  Command: 

-  Generic  72  hour  Caspian  Sea  scenario. 

-  Three  sets  of  missions  for  three  Major  Combat  Operations 
(MCO). 

-  TRAC-LEE  converted  Mean  Miles  Between  Failure  (MMBF)  to 
Mean  Time  Between  Failure  (MTBF)  Reliability  Data. 


The  Model 


*  Dynamic  Sustainment/Logistics  Battle  Command: 


Developer 

TRADOC  Analysis  Center-  Monterey 

Development 

Collaborators 

TRAC-LEE,  TRAC-WSMR,  AMSAA, 
TRAC-FLVN,  CASCOM 

Primary  User 

TRAC-LEE  Sustainment  Analysts 

Primary  Use 

PCS  FY07  OSD  Review 

*  Command-line  run  Java  program. 

*  Simkit  implementation  with  complete  event  graph 
diagram. 

*  Simulates  Class  IX  material  consumption  throughout 
the  course  of  a  scenario. 


NOLH 


*  Nearly  Orthogonal  Latin  Hypercube  (NOLH): 

-Efficient,  “space-fiiiing”,  aii-purpose  design 
generating  a  wide  variety  of  factor  level 
combinations. 

-Fiexibiiity  of  Design  and  Analysis: 

-  Many  fewer  design  points  required  compared  to  traditionai 
designs  such  as  factoriais. 

-  Few  iimits  on  #  of  factors,  #  ieveis,  sampiing  budget. 

-  Aiiows  broad  expioration  of  the  dynamic  soiution  space. 

-  Faciiitates  mathematicai  modeiing  of  muitipie  MOE’s  . 

-  Effective  screening  tooi  for  identifying  threshoid 
settings,  non-iinear  results,  and  multiple  higher-order 
interactions. 


The  Design  of  Experiments  (DOE) 


*  Meta-Modeling:  Build  a  statistical  model  of  the 
simulation  model. 

*  Nearly  Orthogonal  Latin  Hypercube  (NOLH)  and  Critical 
Factors: 

-Design  NOLH  DOE  encompassing  current  DS  input 
parameters. 

-Perform  infiuence  analysis,  based  upon  resuits  of 
baseline  scenario  and  NOLH  design. 

-  Identify  criticai  input  factors. 

-Deveiop  reiationship  between  input  factors  and 
forecasting  methodology. 

-  Identify  criticai  MOE’s  I  MOP’s. 


DOE  Factor  Development 


•  Over  150  Model  Input  Parameters. 

•  Screened  to  10  Input  Factors: 

-  Failure  Dist  Scaling: 

-  The  rate  at  which  any  type  of  faiiure  occurs  onboard  any  vehicie  in 
the  BCT. 

-  Consumabie  Dist  Scaiing: 

-  The  rate  at  which  parts  arrive  to  fiii  materiai  requirements. 
“Veiocitiy  of  parts.” 

-  Auto/Comm  Parameter  Scaiing: 

-  The  number  of  automotive  and  communication  mechanics 
assigned  to  the  two  major  groupings  of  repair  teams-  (1,2,3  CRT) 
and  (Rear  CRT). 

-  Work  Ruie  Limits: 

-  Maximum  number  of  hours  a  mechanic  can  work  per  day. 

-  Prob  Crew  Repair  Damage: 

-  The  probabiiity  the  vehicie  crew  can  repair  damage  to  the  vehicie. 

-  Prob  Crew  Repair  Reiiabiiity 

-  The  probabiiity  the  vehicie  crew  can  repair  reiiabiiity  faiiures  to  the 
vehicie. 

-  Prob  Correct  Diagnosis: 

-  The  probabiiity  the  repair  team  correctiy  diagnoses  the  probiem. 


DOE  Factor  Settings 


*  Factors  Set  to  33  Levels. 

*  Factor  Range  Settings: 

-Failure  Dist  Scaling-  1/10x  to  10x  of  Baseline 

-Consumable  Dist  Scaling- 1/1  Ox  to  lOx  of  Baseline 

-Auto/Comm  Parameter  Scaling- 1/1  Ox  to  lOx  of 
Baseline  (4  Sets) 

-Work  Rule  Limits-  6  hrs  to  16  hrs 

-  Prob  Crew  Repair  Damage-  0.0  to  0.4 

-  Prob  Crew  Repair  Reliability-  0.4  to  0.8 

-  Prob  Correct  Diagnosis-  0.65  to  0.99 

*  MOE  Focus:  Force  Operational  Availability  (Ao). 

*  Future  work  will  include  a  broader  set  of  MOE. 


DOE  Correlation 


ProbCrew  ProbCrew  ProbCorrect  FailureDist  ConsumDist  ForceStruct  ForceStruct  ForceStruct  ForceStruct  Work  Rules 
RepDmg  RepReliab  Diag  ParamScale  AutoScale  CommScale  AutoScale  CommScale  Limit 

Paramscale  (1,2,3  CRT)  (1,2,3  CRT)  (Rear  CRT)  (Rear  CRT) 


ProbCrew  -J 
RepDmg 


ProbCrew  0.009223  1 

RepReiiab 

ProbCorrect  I  0.000924  I  0.004606  M 
Diag 


FaiiureDist  0.011228  0.002649  -0.004974  1 

ParamScale 

ConsumDist  -0.029114  0.008934  -0.027099  0.001149  M  \ 

ParamScale 

ForceStruct  -0.005127  0.001771  -0.016154  -0.001386  0.005126  M  \ 

AutoScaie 

(1,2,3  CRT) 

ForceStruct  0.029021  0.000282  0.006633  -0.008075  -0.003287  0.006641  M  \ 

CommScaie 
(1,2,3  CRT) 

ForceStruct  -0.008467  0.000285  -0.002233  -0.001034  -0.016339  0.009335  -0.001664  1  \ 

AutoScaie 
(Rear  CRT) 

ForceStruct  0.020147  -0.006441  -0.001670  0.001898  0.014490  0.001798  0.015876  -0.006718  1  \ 

CommScaie 
(Rear  CRT) 

Work  Rules  0.030000  -0.002055  0.005395  -0.007321  -0.001437  0.007967  0.999985  -0.001846  0.015084  1 

Limit 
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Results: 

iAg  Alpha  Meta-Model 


1  Summary  of  Fit 

Analysis  of  Variance 

RSquare 

0.996738 

Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

RSquare  Adj 

0.996521 

Model 

2 

0.28835419 

0.144177 

4584.062 

Root  Mean  Square  Error 

0.005608 

Error 

30 

0.00094355 

0.000031 

Prob  >  F 

Mean  of  Response 
Observations  (or  Sum  Wgts) 

0.94959 

33 

C.  Total 

32 

0.28929774 

<.0001* 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.997622 

0.001264 

789.35 

<.0001* 

Reciprocal(FailureDistParamScaling) 

-0.094552 

0.002352 

-40.20 

<.0001* 

(Reciprocal(FailureDistParamScaling)-0.65865)*(Reciprocal(FailureDistParamScaling)-0. 65865) 

0.0048511 

0.00027 

17.95 

<.0001* 

Results: 

iAg  Alpha  Meta-Model 


Initial  Meta-Model 


Scaled  Estimates 


Continuous  factors  centered  by  mean,  scaled  by  range/2 
Term 


Scaled  Estimate 


Std  Error 
0.001258 
0.011641 
0.00662 


t  Ratio 
743.54 
-40.20 
17.95 


Prob>|t| 

<.0001* 

<.0001* 

<.0001* 


V, 


Mean  of  the  Failure  Rate  overwhelmingly  dominates  Ao. 


Results: 

iAg  Alpha  Meta-Model 


factor 

Force  Ao  Total 

Predicted  Force 
Ao  Total 

Lower  95% 

Upper  95% 

1 

0.94173557 

0.94948828 

0.93782525 

0.96115131 

2 

0.97270838 

0.97356676 

0.96192773 

0.98520579 

3 

0.95759706 

0.9403314 

0.92862467 

0.95203812 

4 

0.97615154 

0.97551184 

0.96386873 

0.98715495 

5 

0.94997376 

^  0.956200V 

0.94455731 

0.96784291 

6 

0.9657853 

^\li.965262o\ 

0.95363017 

0.97689401 

7 

0.9823947 

0^^M^3051 

f 

479 

8 

0.97138147 

0.97128>9?^ 

Very  accurate  predictive 

739 

9 

0.97657664 

0.98106293 

capabilities  across  a  broad 

279 

28 

0.78174993 

0.782384^ 

range  of  Ao. 

295 

29 

0.86473149 

0.86§S€f?04 

V 

. J643 

30 

0.97932436 

Jk^1877y 

0.96554036 

0.98883509 

31 

0.92439137 

0.9270983^ 

0.91529725 

0.93889937 

32 

0.90554963 

0.90629716 

0.89427978 

0.91831455 

33 

0.47549603 

0.47541395 

0.45921871 

0.4916092 

Results: 

iAg  Beta  Meta-Model 


Summary  of  Fit 

RSquare 

0.999291 

RSquare  Adj 

0.999128 

Root  Mean  Square  Error 

0.002808 

Mean  of  Response 

0.94959 

Observations  (or  Sum  Wgts) 

33 

Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

6 

0.28909273 

0.048182 

6110.547 

Error 

26 

0.00020501 

7.885e-6 

Prob  >  F 

C.  Total 

32 

0.28929774 

<.0001* 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.9857171 

0.003237 

304.52 

<.0001  * 

ProbCorrectDiag 

0.0183867 

0.003771 

4.88 

<.0001  * 

Reciprocal(FailureDistParamScaling) 

-0.093169 

0.001202 

-77.48 

<.0001  * 

(Reciprocal(FailureDistParamScaling)-0.65865)*(Reciprocal(FailureDistParamScaling)-0. 65865) 

0.0046751 

0.000138 

33.78 

<.0001  * 

Consumable  Dist  Factor  Scaling 

-0.000985 

0.000169 

-5.82 

<.0001  * 

(Consumable  Dist  Factor  Scaling-5. 0503)*(Consumable  Dist  Factor  Scaling-5.0503) 

0.0002627 

6.431  e-5 

4.08 

0.0004* 

(ProbCorrectDiag-0. 77061  )*(Consumable  Dist  Factor  Scaling-5.0503) 

-0.003749 

0.001066 

-3.52 

0.0016* 

Results: 

iAg  Beta  Meta-Model 


Residual  by  Predicted  Plot 

0.006 

0.004 

^  §  0.002 

8  ^  0.000 
1°  ro 

o  -0.002 

-0.004 

-0.006 
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Improved  Meta-Model 


Scaled  Estimates 


Continuous  factors  centered  by  mean,  scaled  by  range/2 
Term 
Intercept 
ProbCorrectDiag 

Reciprocal(FailureDistParamScaling) 

(RS(|N^cal(FailureDistParamScaling)-0.65865)*(Reciprocal(FailureDistParamScaling)-0. 65865) 
ConsumaclNmst  Factor  Scaling 

(Consumable  Dis^S^^r  Scaling-5. 0503)*(Consu?^^le  Dist  Factor  Scaling-5.0503) 
(ProbCorrectDiag-O.TTOTHjlJfConsumable  Dist  Facton^aling-5.0503) 


Scaled  Estimate 
0.9335442 
0.0040451 
-0.461185^ 
0.114552 

0.0q^361 
04082 


Std  Error 

t  Ratio 

Prob>|t| 

0.000811 

1150.63 

<.0001* 

0.00083 

4.88 

<.0001* 

0.005952 

-77.48 

<.0001* 

0.003392 

33.78 

<.0001* 

0.000838 

-5.82 

<.0001* 

0.001576 

4.08 

0.0004* 

0.001161 

-3.52 

0.0016* 

Results: 

iAg  Beta  Meta-Model 


factor 

Force  Ao  Total 

Predicted  Force 
Ao  Total 

Lower  95% 

Upper  95% 

1 

0.94173557 

0.94730071 

0.94125295 

0.95334848 

2 

0.97270838 

0.96838136 

0.96226359 

0.97449912 

3 

0.95759706 

0.955104 

0.94840414 

0.96180386 

4 

0.97615154 

0.97423641 

0.96741987 

0.98105295 

5 

0.94997376^ 

0.95304409 

0.94710902 

0.95897916 

6 

0.9657853 

^^^0.962055^ 

0.95605447 

0.96805751 

7 

0.9823947 

0?9?§19169^ 

^  0.97127004 

0.98511333 

8 

0.97138147 

0.969^^3 

A . . 

9 

0.97657664 

0.976431^ 

sA  ( 

Superior  predictive 

28 

0.78174993 

0.78020106 

capabilities  across  a  broad 

29 

0.86473149 

0.8700869> 

range  of  Ao. 

30 

0.97932436 

( 

L 

31 

0.92439137 

.^^^^92543JfM 

0.91931045 

0.93156664 

32 

0.90554963 

0.90202376 

0.89582181 

0.9082257 

33 

0.47549603 

0.4755157 

0.46735401 

0.48367739 

Results: 

Observations 

•  Lessons  Learned  from  Meta-Model: 

-  Passes  Common  Sense  test. 

-  The  overwhelming  factor  which  drives  Ao  is  the  failure 
rate. 

-  Low  failure  rate  implies  high  system  reliability  as 
measured  by  high  Ao. 

-  Demonstrates  that  component  reliability  drives  system 
reliability  for  this  model,  operating  in  this  range  of  inputs. 

-  Demonstrates  value  of  understanding  underlying 
relationships  within  model. 


The  resulting  equation  is  a  “quick  and  dirty” 
substitute  for  the  simulation. 


The  Way  Ahead 


*  Focused  Range  Model  within  likely  parameter  operating 
ranges: 

-  Broad  spectrum  expioratory  anaiysis  defined  how  the 
modei  worked. 

-Future  work  wiii  focus  on  more  plausible  parameter 
inputs. 

*  Time  Series  Analysis: 

-Meta-Model  for  MMH  and  Parts  Fill  Rate. 

-Determine  appropriate  COP  “snap-shot”  rate  for 
determining  time  segments  used  in  forecasting. 

*  Compete  four  basic  forecasting  methodologies: 

-  Original  stepwise  regression  model. 

-Exponential  smoothing. 

-  Part  period  (3-period)  moving  average. 

-Auto-Regressive  Moving  Average  (ARMA)  /  Auto- 
Regressive  Integrated  Moving  Average  (ARiMA). 
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